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Lecture 06: 
Harris Corner Detector 


Reading: T&V Section 4.3 
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Motivation: Matchng Problem 


Vision tasks such as stereo and motion estimation require 
finding corresponding features across two or more views. 


we 
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CSE486, Penn State Motivation: Patch Matching 


Elements to be matched are image patches of fixed size 
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ot all Patches are Created Equal! 


Inituition: this would be a good patch for matching, since 
it is very distinctive (there is only one patch in the second 
frame that looks similar). 
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ot all Patches are Created Equal! 


Inituition: this would be a BAD patch for matching, since 
it is not very distinctive (there are many similar patches 
in the second frame) 
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What are Corners? 


+ Intuitively, junctions of contours. 

+ Generally more stable features over changes of viewpoint 
+ Intuitively, large variations in the neighborhood of the point 
in all directions 

* They are good features to match! 
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Corner Points: Basic Idea 


* We should easily recognize the point by looking 
at intensity values within a small window 


e Shifting the window in any direction should yield 
a large change 1n appearance. 


ga, Appearance Change in 


Interactive Neighborhood of a Patch 


"demo" | ۳۶ ۳ 


Robert Collins 


CSE486, Penn "Harris Corner Detector: Basic Idea 


"flat" region: "edge": “corner”: 
no change in no change along significant change 
all directions the edge direction in all directions 


Harris corner detector gives a mathematical 
approach for determining which case holds. 
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Harris Detector: Mathematics 


Change of intensity for the shift [u,v]: 


^ Window ` 
« function ,/ 


Window function W (X, y) E 


| in window, 0 outside Gaussian 
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Harris Detector: Intuition 


Change of intensity for the shift [u,v]: 


Intensity ` 


For nearly constant patches, this will be near 0. 


For very distinctive patches, this will be larger. 
Hence... we want patches where E(u,v) is LARGE. 
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Taylor Series for 2D Functions 


f(x +u, y+ v) — f 27 y) + uy fx (x, y) +y f (x. y) + 


First partial derivatives 


Ay | . NW ۱ 
yi [i8 fy) Hwy [+ 
Second partial derivatives 


u xy wv fas x, y) + uv? fy (x, y) 4- v? foy (x, y)| 


Third partial derivatives 


| 
3! 
+ ... (Higher order terms) 


First order approx 


f(x +h Oy + v) > Tx. y) +u f (x, y) +y ip (x, y) 
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csse remsa Harris Corner Derivation 
Y, Ge uy ty) = GE 
e » IF (x, y) bul, + ۷], — Hx, y) First order approx 
— > + 2uvl JA, + VE 


— » ۳ v | E A M Rewrite as matrix equation 
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Harris Detector: Mathematics 


For small shifts [u,v] we have a bilinear approximation: 


where M 1s a 2x2 matrix computed from image derivatives: 
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7 "TIntuitive Way to Understand Harris 


Treat gradient vectors as a set of (dx,dy) points 
with a center of mass defined as being at (0,0). 


Fit an ellipse to that set of points via scatter matrix 


Analyze ellipse parameters for varying cases... 
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X derivative Input image patch 


Y derivative 
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CSE486, Penn “Plotting Derivatives as 2D Points 


The distribution of the x and v 
derivatives is very different for 
all three types of patches 


Corner Linear Edge 
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""Fitüng Ellipse to each Set of Points 


The distribution of x and v | 
derivatives can be characterized |" 
by the shape and size of the 
principal component ellipse 


" A1—A2 = small 


L 1 i 
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Corner 


A1—A2 = large 
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Classification via Eigenvalues 


Classification of 
image points using 
eigenvalues of M: 


À, and ۳ are small; 
É is almost constant 
in all directions 
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Corner Response Measure 


Measure of corner response: 


(kis an empirically determined constant; k = 0.04 - 0.06) 
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me Corner Response Map 
(0,0) lambdal ۲ 
R=28 
1 R=65 
d 
pa R=104 
R=142 


R= det M — k (trace M ) 
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CSE486, Penn State Corner Response Map 


R<0 “Edge” 
lambdal 


|R| small 
“Flat” 


* R depends only on 
eigenvalues of M 


+ R is large for a corner E Bess 
| 5 

+ Ris negative with large & 155 R=104 

magnitude for an edge / p an 

* |R| is small for a flat im 

region ra 


«SPA, 0 > A 
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CSE486, Penn State Corner Response Example 


> E'N 


Harris R score. 


Ix, Iy computed using Sobel operator 
Windowing function w = Gaussian, sigma=1 


Corner Response Example 


Threshold: R < -10000 
(edges) 


Corner Response Example 


F E 
5 m J^ då 
Threshold: > 10000 
(corners) 


Corner Response Example 


| = = 


Threshold: -10000 <R < 10000 
(neither edges nor corners) 
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CSE486, Penn "Harris Corner Detection Algorithm 


1. Compute x and y derivatives of image 
Iz = Gł * ] Y= GY + I 


2. Compute products of derivatives at every 
pixel 


Ir? = Top. I, 1,2 = Iy.Iy Izy = Iy.ly 


3. Compute the sums of the products of deriva- 
tives at each pixel 


S2 = Gor * I,2 Sy2 = Gor * I Sry = Gor * Iry 


4. Define at each pixel (x,y) the matrix 


S,o(z,y) Szy(m, y) 


H(z, y) = Say(x y) Sy2 (x, y) 


5. Compute the response of the detector at 
each pixel 


R = Det(H) — k(Trace(H))? 


6. Threshold on value of R. Compute nonmax suppression. 


